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ABSTRACT 

Researchers have only recently recognized that 
assertions by various educators that they have implemented 
* educational innovations do not a\l mean the same thing or bear the 
same weight, according to the authors of this report. Efforts made by 
the Research and Development Center for Teacher Education alt the 
University of Texas at Austin tOoformalize research on the 
implementation of innovations led to the development of two 
measurement concepts. The first of these, used jwhen assessing the 
degree to which innovations have been implemented, is called levels 
of use of the innovation; the second, establishing a perspective from 
which to define an implemented innovation's operational form, is 
called innovation configurations. This report discusses several 
alternative perspectives frop which implementation can be viewed, 
then details the characteristics of the innovation configurations 
approach. Several conceptual, methodological, and practical issues 
emerging from this way of defining the status of innovations are 
explored in the report's final section. (Author/PGD) 
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During the last five years change process researchers have focused in on 
the implementation phase. There has been' a concentrated effort to develop di- 
mensions and conceptual frameworks that will help in understanding and describ- 
ing change process dynamics as they unfold during this important phase. Impli- 
cations of this research bear directly upon issues that confront evaluators, 
researchers in other fields and practicing change facilitators. 

Until quite recently implementation of an innovation, a new program or 
procedure, was assumed to have taken place if an administrator, or in some cases 
the supposed user, stated that the innovation was in use. In the 1960's there 
were attempts to develop "teacher proof 11 innovations, which would automatically 
be implemented upon delivery without local site modification. Recently, in 
contrast, there has bee^n a great deal of attention paid to the concept of 
"mutual adaptation," where the innovation is adapted by and for each user.. The 
concept of "fidelity" has been an issue all along. Yet, clear 'resolution of 
implementation related conceptual and methodological problems has not yet been 
achieved for researchers, evaluators or practitioners. 



*The research described herein was conducted under contract with the 
National Institute of Education. The opinions expressed are those of the 
authors and do not necessarily reflect the position or policy of the National 
Institute of Education. No endorsement by the National Institute of Education 
should be inferred. *> 

^Paper presented at the annual meeting of the American Educational 
Research Association, April 1981, Los Angeles. ■ 
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As a part of research on the change process using the Concerns-Based Adop- 
tion Model, staff at the Texas R&D Center have been grappling with implementa- 
tion issues. The mission of the research project is development of concepts 
that will help in furthering understanding of the change process and contribute 
to more effective research, evaluation and change facilitation efforts. 

. A very early issue in this research was determining if various supposed 
innovation users were in fact using the program or process (i.e., the innova- 
tion) that was under consideration. Field staff quickly observed that the tes- 
timony of administrators and s even supposed users was not necessarily a valid 
indicator of how much of a particular innovation was in fact being used. There 
was a pressing need for a framework, procedures and criteria to use in determin- 
ing when the innovation was actually in use. After much discussion, field 
experience and exploration it became clear that two basic questions needed to be 
asked of each subject.- 

1. Is it, the innovation, being used? 

2. What is "it"? 

Although these two questions seem obvious in hindsight, their initial specifica- 
tion and the determination of how to collect information to answer them was not 
as straight-forward as might appear. e 

Is It Being Used? 

In Texas Center ^research, the first question was addressed through the con- 
cept of Levels of Use of the Innovation (Hall, Loucks, Rutherford & Newlove, 
1975). The question "Is it being used?" implies, that use is a dichotomous vari- 
able, that either a person is or is not using the innovation. However, our re- 
search suggested that the answer was more complex, and from that complexity 



eight different Levels of Use were derived. This concept distinguishes between 
three different levels of nonuse and five different user levels. The. Levels of 
Use describe in operational terms how users change their behavior with the inno- 
vation as they move from no innovation related' behavior , to early mechanical use 
and to later more sophisticated use. The concept also tak.es into account the 
kinds of adaptions or changes a user makes „in use of the" innovation or in the 
innovation itself. . 

The Levels of Use concept has several implications for evaluation person- 
nel, researchers, and change facilitators. Contrary to what was frequently' 
assumed in the past, the claims of administrators and, in~many cases, even 
teachers about use of an innovation are often inaccurate. In one study, for 
example, it was found that only 80 percent of the teachers in a treatment group 
were in fact using the innovation; twenty percent were not. Iff the so-called 
"comparison group," 49 percent of the sample were using the innovation (Hall & 
Loucks, 1977). This leads to the phenomenon that Charters and Jones (1973) hav t e 
referred to as the evaluation of "non-events". All too frequently it appears 
that researchers as well as evaluators have simply assumed the existence of a 
treatment group and a comparison group when in fact* all the users were not in 
one group and nonusers in the other group. At this point it' seems clear that 
the question "Is it being used?" does need to be addressed in research and 
evaluation efforts. The concept of Levels of Use is one demonstrated way to 
address this question. , „ 

What Is It? 

o ... 

, The second question, "What is it?" is more problematic. This question 
again has been addressed in the literature from various orientations. The 
answer to "What .is s 'tt?" seems to vary depending upon the orientation that is 



used to address the question. This compounds 'the problem for researchers, 
evaluators, and change facilitators alike who must know the innovation when they 
see it. defining what it is is not a simple task. 

* In the next section of this paper some of the alternate perspectives that 
have been used to define "it," the innovation, are explored. In a later section 
our emphasis, describing the innovation in terms of its operational form, will 
be explored. We have coined the term, Innovation Configurations, to represent 
this orientation. The final section of the paper is ued to explore several 
conceptual, methodological and practical issues that emerge from this way of 
defining the innovation. 

Which Orientation? 

i 1 * . 

Very different orientations have been used to define innovations in the 

literature. The variables that are assessed to determine implementation and 

treatment effects can vary considerably depending on the orientation that is 

selected. Further, the use/nonuse question is compounded by the orientation .„ 

that is taken for defining the innovation. The following briefly describes some 

of the different orientations that have been used to describe innovative pro- 

2 ... 0 

grams, procedures *and practices. Note that each orientation for defining the * 
Innovation requires assessment of different variables and attributes and in, some 
cases different subjects would be assessed. . 

Perceived Attributes J 

One of the best known orientations for defining the innovation is that 
developed by Rogers & Shoemaker (1971). They describe the innovation in terms 
of how it is perceived by prospective adopters. They propose five attributes of 
innovations: (1) relative advantage, (2) compatibility, „(3) complexity, (4) 
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•c . ■ .... 

tri'al ability, and (5) observability. With this orientation it does not matter 

il 

w an absolute "sense, what the innovation actuXjy is* but^ rather how it is per- 

~V , ■ 

ceived by potential users. Thus, those innovations that appear to have a rela- 
tive advantage, to be compatible with present practices, not too complex,' easy 
to try out and yield results that are easily observed, are more likely to be 

o 

adopted. From a diffusion or communication of innovations point of view this 
way of addressing the characteristics of an innovation makes good sensfe. For 
those who are interested in implementation, however, this orientation is not of 
much assistance since it does not provide information about the innovation 
itself and what use actually entails.^ 

Philosophy 

Many innovative programs and practices are described in terms of their 
philosophical orientation, the values,' underlying assumptions and beliefs that 
are associated with the innovation. From this orientation the innovation is 
described in terms of the fundamental beliefs of the innovation' developers. 
This orientation for defining an innovation is particularly useful at the ini- 
tial selling phase, since the innovation can be assessedjn terms of how closely 
it fits or competes with the value positions of potential adopters. Again, for 
evaluators, researchers and change facilitators who are concerned about imple- 
mentation, having merely a description of the philosophy of an innovation does 
not help in assessing whether or not it is in use. There is apt to be a large 
gulf, if not an absolute lack of correlation between espoused philosophy and* 
actual practice. 

Goals and Outcomes 

More recently, innovations have been described in.terms of their c goals and 
outcomes. With this orientation, the overall goals of an innovation ar»e empha- 
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sized, frnquently in broad-sweeping terms. In many cases more specific objec- 
tives and promised -effects of the innovation are also highlighted: Due to the 
push for. ""validation" of programs and strong emphasis upon the identification 
and specification of outcomes that can be associated with an innovation thfs 
orientation seems to be of increasing importance, /From an implementation per-, 
spective, this orientation also does not make clear exactly how the innovation 
was used to achieve the outcpmes that are associated with it. 

Implementation Requirements . 

Another common* way of describing an innovation *is in terms of its imple- 

* ■ '«>■'. 

mentation requirements. This is an orientation in which potential adopters are? 

naturally interested.. Those concerned about high fidelity implementations also 
■ 

place special emphasis upon the description* of implementation "requirements. 
Thus, an innovation" is described in terms of those steps*, procedures and re- 
sources that are needed in order for it to be adopted. With this orientation an 
innovation description may include the required classroom space,- the permitted 
teacher/student ratio, the name of the textbook and tests and special inservice 
sessions that must be attended. The concern from an implementation' 4 perspective 

o # 

is once again, having taKen all of the steps to address implementation require- 
ments does not indicated or how the innovation^ being used. Requiring the 
presence of five microscopes indicates nothing about how those microscopes are 
to be used when the innovation is in operation. ^_ — 

o 

Functions ^ 

Another orientation advanced by Treadway (1980), is description of an inno- 
vation in terms of its functions. The question becomes, "Are the various func- 
tions of the innovation being, carried out?" If so, then one can say that the . 
innovation is in place. From a conceptual point of view, this orientation has a 



great*~deal of appeal. Operationally however, it appears to require 4 great deal 
of skill and effort to identify and describe the various functions. Further, it 
is conceivable that the unique properties of a particular innovation could tje - 
lost in the description of more, generic functions. Determining those features 
that are exclusively *part of the specific innovation under consideration could 
be quite difficult. At a mone global level, this is not an issue. However, if 
one is concerned abolit implementation of a specific innovation in a specific 
context, then maintaining the innovation's identity wi.thiij the functional analy- 
sis may become problematic ^ * 



Behaviors « <- 

I . For change facilitators, evaluatbrs* and researchers none of these oriehta-, 
ttons focus on identification and description of the actual treatment that each 
program user delivers. This is the problem that was encountered in the early 

Levels of Use research. Just because teachers or professors have taken the 

• - v 
training, have the materials in the classroom, are able to espouse the philo- 

* ** <> 

sophy of the 'innovation and have "adopted" it from the point of view of per- 
ceived attributes, does not teH what they are doing. In fact what" might be - 
observed in one classroom could be inconsistent wUh what was observed in * 

another classroom. The configurations of * the innoyation would vary yet all 

<> ... 
*•*... 

could* be "users" of the innovation in name. 0 



Knowing what is actually being used in the name of a particular innovation 
is important for a number of reasons. Fullan and Pomfret (1977) mention 
four: - 

(1) to be clear about exactly what has changed as a result of an 
innovation effort, * 

(2) to understand some of the reasons why so many educational changes 
fail to become established, c 
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: (3)* to avoid ignoring the important phase of implementation, and 

(4) to allow interpretation .of , learning outcomes and relating these 
. " to possible determinants, 

Jn order to, -understand the complex process of change, itsoutcomes and its 

determinants, it is clearly- necessary to monitor o the changes in practice that 

are actually occurring. * a ■'• * ■ 

Measuring Implementation , n 

The issjie of how to measure what is actually being implemented has been 

given much thought in 'recent years, especially in terms of describing behaviors 

( Lei nharcft^ 1980) . In thejr evaluation studies of Follow Through Models, 

Stallings and Kaskowitz (1974) were among the first to* establish a set of pro'ce- 

du res- to .verify that an innovation was actually in use before collecting eval.ua- 

tion -study data. In thei/ study, innovation developers wer$ surveyed for de- 

Ascriptions bfywhat thje innovation should look 'like -in use. The presence 'of 

* * ** . 

these characteristics .in study classrooms was then documented. 

Evans and Schaffler (1974) also concentrated on the. innovation and how it 
was being implemented when they evaluated the Individually Prescribed Ins^ruc- 
tion, (IPI) mathematics program. ' In their. work they attempted to assess which of 
eleven devteloperTidentifi^'irtnovation categories were in use at eaclj study • ; 
school. They discovered that not only did the^ degree of implementation of the 
innovation (i.e., 'the number of categories observe?!) vary among schools, but 
that there were even "differences in the categories.* 

; The problems of how to measure the degree of implementation of an innova- 
tion are just beginning to be understood. Gephart (1976) has identified Several 
classes of measurement problems and more recently Owens and Haenn (1977) Jiavfe . 
developed a cl^ ar summary of difficulties. The latter autjiors also provide 



il lustrations of how they have handled the problems .in ttieir evaluation of 

v r 

Experience -Based Career Education (EBCE). 

* Work done at the Texas R&D Center in elementary schqols, where the focus 
was on studying •implementation of various educational innovations, cle^ly docu- 
mented the need to better understand what users" were actually doing with an 
innovation, as well as to explore procedures to assess their varied use. For 
example, in one study Center staff found many teachers who said they were team- 
ing, buWtheir descriptions indicated wide variation. One team might consist of 
two teachers who met once a month to share lesson plans, and kept their own 
classes intact through the school year. Another team would have 4 teachers, 
with time every Thursday afternoon ' f c? teacher planning, and students constantly 
revolving from teacher to teacher for different subject matter areas. Both sets 
.of- teachers were "tetters-. N "Yet, the conf iguration^ of teaming that they were 
using were" extremely different! This led to the proposal of another orientation 
for defining the Innovation, that of Innovation Configurations, (Hall & Loucks, 
1978).^ , ■ . V < 

The Conceit of Innovation Configurations 

k * 

y 

The-,emphasis of Innovation^ponfigurations is upon describing the. opera- 
tional form of the innovation as it is being used by^each person. This includes 
the actual behaviors and roles of the people, teachers and students, how mate- 
rials are used, how special procedures ar£ performed and' what the interactive 
strategies, relationships and processes are. 

For any innovation, major components can be identified. For example, the 
components of :an individualized program in mathematics might include how 
materials are used, (2) how the students are grouped, (3) how students are 
tested, and (4)*what is done wi.th test results. Or a procedure such as the J . 

... M . ... ■» -• - - ■ - " >\. 
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direct instruction model, which has been developed by classroom researchers could 
be described in terms of such components as (1) academic content coverage, (2) 
student engagement, (3) role of the teacher, (4), grouping, and (5) management 
procedures. 

For each component, variations in how that component can be used are iden- 
tified, a "For example, for each of the innovations identified above grouping of 

c * 

studentship a component. Variations on the grouping component could include (a) 
one large heterogeneous group, (b) one large homogeneous group, (c) several 
small groups, or (d). individualization. Variations for other components can 
£lso be. identified (See Figure 1). 



Figure 1 

Innovation Components and Variations 



Component 1: Grouping" 

Variation 1: One heterogeneous group 

Variation 2: One homogeneous group 

Variation 3: Several small groups 

Variation 4: Individualized 

Component 2: Materials Usage 

Variation 1: Uses textbook only 
Variation 2: Uses program materials only 
Variation 3: Uses a combination of materials 



Across classrooms within a particular building, across a particular 
school district, and clearly across different sites in different school dis- 
tricts, there is likely to be considerable variation in how each, component is 
actually implemented by each user. Thus* in terms of the component, grouping of 
students, the innovation could vary in significant ways. Yet, all teachers 
should be "users" of the innovation. 
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The challenge for the researcher, evajuator, or change facilitator is first 
to determine which components of the innovation are in fact being used and 
secondly, to determine which variations of each component are in use. Compo- 
nents usually represent descriptions of the materials, role^ and style of the 
users as they are interacting with the processes and procedures of the innova-< 
tion. Once information about these components is documented then other ques- 
tions might be addressed. Why are certain component variations observed and 
cfthers not? What outcomes are associated with the innovation and which are not? 
Why and how users adapt innovations in particular circumstances and contexts can 
be addressed. 

Assessing Innovation Configurations 

The key to identifying Innovation Configurations is v to first determine 
the components and the component variations that describe the innovation in use. 
The degree of specificity and the complexity needed is best determined by con- 
sidering the use to be made «of the resultant information. An innovation devel- 
oper may emphasize ten components while the practitioner may consolidate these 
into three or four. Further, the innovation developer may, and often does, 
tolerate less variation in each component than does the practitioner. 

For Texas R&D Center research a basic procedure has; been developed for 
identifying the components and component variations of each innovation to be 
considered (Heck, Stiegelbauer, Hall & Loucks, 1981). The first step entails 
review of the materials that are available that describe and constitute the 
innovation. Then, if it is possible, the developer or another individual who 
represents expertise in use of the innovation is interviewed. Following this a 
sample of users representing a range in use of the innovation are observed and 
interviewed. 
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Throughout this process the major components of the innovation and how each 
of these components can vary are being identified. These are summarized in an 
Innovation Configuration* Checklist . Information to complete a checklist may be 
collected through procedures such as observation, interview and questionnaire. 
For a particular subject the combination of component variations that are check- 
ed on the checklist represents the configuration of the innovation that that 
particular person is presently using. A sample checklist appears in Figure 2. 

One important point worth reemphasi zing is that the Innovation Configura- 
tion Checklist is distinct and different from the collection of data to complete 
the checklist. The checklist is a synthesis and record of the findings. 
Gathering the information to complete the checklist on each user is a separate 
step. Also, determining why particular component variations and configurations 
are in use and others are not is a separate question from determining what the 
configurations are. 

Issues and Implications of Innovation Configurations 

The Innovation Configuration perspective for describing what the innova- 
tion is has been highlighted briefly. In this approach the emphasis is placed 
upon identifying and describing the various operational forms of an innovation 
as implemented by users. This orientation is particularly important for those 
conducting research and evaluation studies and is also important for change 
facilitators, who have the role of encourcging implementation of various innova- 
tions. Without innovation configuration information it is all too easy^to make 
inaccurate inferences about how the innovation is being used. This can greatly 
endanger potential studies because of inaccurate interpret atfon of study re- 
sults; it can also lead to change facilitators designing and delivering staff 
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Figure 2 
Tutoring Program Checklist 



1. Materials and Equipment | 

(1) | • (2) 

At least 5 different program | At least 3 different program 

materials are used with each materials are used with eachj 

child each session. | child each session. 


(3) 

Less than 3 different pro- 
gram materials are used with 
each child each session. 


2. Diagnosis . 

• (1) (2) 

Children are diagnosed Children are diagnosed 
individually using a | individually using teacher 
combination of tests and judgment only, 
teacher judgment. | 


(3) 

Children are not diagnosed 
individual ly. 


3. Record-Keeping . 

Individual Record Sheet is 
used to record diagnosis and 
prescription. . 


(2) / „ 

No Individual Record Sheets 
are used. * 


4. Use of Teaching Technique | 
(1) | 

Continually readjusts task | 
according to child needs; . 
uses rewards to reinforce 
student success. . 


(2) , ( 

-Does not continually readjust 
task according to child 
needs; does not use rewards. 


5. Grouping | 
(1.) | 

Children are taught in | 
pairs. . 


(2) 

Children are not taught in 
.pairs. 


6. Scheduling | 

(1) | (2) 

Children are taught for .30 | Children taught for 30 min. 3' 
minutes 3 times per week. . times per week, time for each 
Eachsession is equally | child and each task varies 
divided between children. . slightly when necessary. 


(3) 

Children not taught for 30 
min. per week 3 times per 
week, or time for each child 
and each task varies mark- 
edly c or is not considered. 



CODE: Variations to the right are unacceptable; variations to the left are acceptable. 

* 

Variations to the left, are ideal, as prescribed by the developer. 

of. " . 

16 -■ 

■ - - - - - ^naatBi — - a - m '^s^^uu^m^^^^^mm '—~*~*~~~>-"*-.-^- — — 
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development experiences or other interventions that are not congruent with the 
needs and present practices of their clients. 

In the remainder of this paper, as well as the other papers that are a part 
of this symposium (Heck, 1981; Melle & Pratt, 1981 ; Trohoski , 1981) some of the 
issues that are raised by looking at innovations from this perspective are ex- 
plored. Each of these issues represents an area where there has been interest- 
ing and extended dialogue and debate. In some instances the Texas R&D Center 
staff have developed some fairly definite directions and opinions. With other 
issues their continue to be "friendly and frank discussions." We invite others 
to join in^this exploration and discussion. All who are concerned with the 
change process have to face these questions and prohlems and develop explicit or 
implicit answers. First, some of the conceptual issues that have arisen, are 
discussed, then some research and evaluation issues and finally some implica- 
tions ftjjr change facilitators. - 

Conceptual Issues 

Although product or material -based innovations are typically used for 
illustration purposes, the concept of Innovation Configurations can be applied 
to all types of innovations. The concept applies equally well to. process inno- 

a 

vations, non-classroom innovations,, non-instructional innovations and group use 
and organizational innovations. In all cases, the essential questions are the 
same: "What does the innovation look like in operation?" and "What are the 
major components and component variations that can be observed or that must be 
considered?" Regardless of the situation there are some interesting conceptual 
questions that arise out of this orientation. 
1 • Are all components of equal importance ? 

• 17 



Clearly, with any innovation, all innovation components are not of equal 
importance. One useful way to consider these has been to identify critical 
components and related components. The critical components are those that must 
be in place in order for the innovation to be considered in operation. The 
related components are these that represent embellishments. They may contribute 
to the overall robustness of the innovation or lead to additional effects that 
are nice to have but are not necessary for basic use of the innovation.* 

Interestingly, this relates to Treadway's (1980) work on the functions of 
innovations. Extrapolating from what Treadway has said, it seems conceivable 
that certain different Innovation Configuration components or component varia- 
tions could be "functionally equivalent". It is conceivable that certain com- 
ponent variations may have the same effects as others. Such situations are in 
part what lead Treadway to advocate that evaluators need to assess functions 
rather than to require that particular components be in place. 

However, this clearly adds another dimension to the evaluator's role. 
Before a particular component or component variation can be judged as equivalent 
to another component or component variation someone must answer the equivalence 
question. Thus, the evaluator would have to run tests of equivalence across 
various component variations, before each could be judged as equivalent. 

By using the concept of Innovation Configurations this question can be 
avoided, the results of the initial summative evaluations of the innovation or 
some agent's judgement can provide the basis for determining what operational 
components are associated with the desired outcomes. Clearly, this too is 
problematic. However, the encouraging note is that between work at the Texas 
Center and that of Treadway, the question can be explored both conceptually and 
empirically. 

2. Whose perspective is used to define the components? 
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One of the basic questions that has to be addressed *in any study is the 
point of view used to determine the components and component variations of a 
particular innovation. Depending upon the source that is selected, not only is 
it % likely that the terminology used will be different, but also it is quite con- 
ceivable that very different components will be identified 0 . For example in ohe_ 
Texas R&D Center study the developer of the innovation placed strong emphasis 
upon the components of "use of tests" and "teaching to the objectives," while 
teachers focused upon the record keeping system and the grouping of children* 
The development of Innovation Configuration components, component variations and 
the composite checklist must take into account the perspective that is going to 
be used to make these specifications. The optimal situation would be a consen- 
sus in terminology across change facilitators, developers, users, and evalu- 
ators. However, this is not always possible. Consensus, or lack of it in a 
particular case, is more a consequence of management of the change process than 
it is an inherent dilemma within the concept of Innovation Configurations. 

3. How do you handle adaptation- ? 

Regardless of the perspective that is taken there will be a point where the 
component variations are so changed that they move outside of the realm of what 
one would classify as innovation-related behavior .' This point of drastic 
mutation is an important one for ,evaluators, researchers and change facilitators 
to identify. It becomes important in" specifying the use/nonuse decision point 
which is so crucial to determining the existence of treatment and corrtrol 
groups. It is also important to the design* of staff development and other 
interventions to support use of the innovation. If certain component variations 
can be identified as beyond the. point of drastic mutation, that is outside the 
realm of acceptable use of this innovation, then all parties can be aware of 
this and appropriate actions can be taken. 
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Note on the Innovation Configuration Checklist in Figure 2 that some com- 

o 

ponent variations, those to the left of the solid line, have been identified as 
acceptable. Tose to the left of the dotted line represent ideal variations. 
TJiis kind of indication is useful if the checklist is to help answer questions 
such as: is the innovation being used in an acceptable configuration? 

One of the interesting aspects of the point of drastic mutation is that 
rather than this being a point, as soon as multiple sources are checked it fre- 
quently becomes an area of drastic mutation. In many cases, there is not con- 
sensus about which component variations are acceptable and unacceptable prac- 
tice. Clarification of this issue alone could greatly facilitate change efforts 
and studies that are being done of them. 

4. When and how does one determine fidelity, of implementation? 
The identification of Innovation Configuration components and Component 
variations might imply to some a fidelity perspective, since the emphasis is 
upon describing what use of the innovation actually entails. However, it Js 
possible to be purely descriptive by simply describing each variation and not 
distinguishing ideal and acceptable variations. Again, whether or not this is 
.done depends on the use to which the data are to be put, and the evaluation or 
research questions to be answered. , 

On the other hand, the concept of Innovation Configurations can also help 
us operational ize the notion of fidelity. Basicajly, fidelity could bedefined 
as replication of the essential components of the innovation. However, this 
does not determine the* range of variations of each component that will be 
accepted. It is conceivable that in one case fidelity may include a large range 
of variations of each component while in another instance, fidelity may'by defi- 
nition be restricted to one set of configuration variations. How closely fidel- 
ity must fit the developer's ideal model is one that has to be determined for 
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each particular setting. With some developers a large number of configurations ° 
are encouraged, while with other developers only one or two configurations are 
considered acceptable. In other instances very sophisticated configurations may 
in fact represent nonuse of the innovation. 

t 6 .. ' 

C 

Fidelity then becomes a judgement call made by one or more sources.* The 
judgement may be made by the developer, it. may be.made by the change facili- 
tator, or it may be left to the evaluator. In other instances users determine 
individually what fidelity means. The point that has emerged from work with the 
Innovation Configuration concept is that this question needs to be answered for 
each study. If it is not answered explicitly, it will be implicitly. 

5. Are there generic components that cut across innovations? 

Logically it would seem that for various types of innovations some of the 
same components would be identified. These "generic" components might be appli- 
cable to most innovations. Having a list of the'se would aid evaluators, re- 
searchers and change facilitators in identifying the components of their parti- 
cular innovations. Fullan and Pomfret (1977) have* proposed one set of compo- 
nents that might be a start. in this direction. These are; materials, struc- 
ture, role/behavior, knowledge and understanding. Leithwood and Montgomery 
(1980) have also given a great deal a of thought to this in terms of possible com- 
ponents of curriculum innovations. In a study which identified components for 
45 different innovations, (The NETWORK, 1981) components in the following cate- 
gories were identified repeatedly: 

Materials Grouping 

ContentX Sequencing/Pacing 

Prescriptive Activities Scheduling 

Teacher/Student Interactions Coordination 

Affective Strategies Testing 

Instructional Activities. Ongoing Assessment 

Student Activities Screening 

Record Keeping * Program Evaluation 



19 



At this time there are not a clear set of generic components, especially 
across various classes of innovations. However, in theory it is pleasing to, 
think that there probably are basic components that could be identified and used 
to help structure the description of specific innovations. If generic compo- 
nents could be identified this would simplify the problem of cross-innovation & ' 
comparisons. 

o " 

6. How do configurations evolve? 

The assessment of the Innovation Conf iguration of each user is a .time- 
dependent phenomenon. Innovation configurations represent a snapshot of user 
practice. It appears from the studies done to date that there is an evolution 
to the configurations that a teacher uses over time. In fact it appears fronra 
couple of analyses that early users are more .apt to use particular configura- 
tions of an innovation and persdns with more experience with an innovation can 
be associated with other configurations, while some configurations can be^found 
across the experience range. , 

Clearly, contextual factors and change facilitator interventions are likely 
to affect which configurations are present at a point in time. If implementa- 
tion is to be understood through research and evaluation studies, longitudinal 
design^ are a must. 

7. What about the size of an innovation? 

* Another interesting dilemma is attempting to develop' means for comparing 

different innovations. How can study findings and implications of change 

• <* ** . 

"efforts be contrasted across several innovations? One promising concept at this 
point is that of size 3 . Innovations come in different siz?s. The dimensions of 
size may include variables such as the number of components, the amount of 
trauma that implementation entails, the amoufit of resources and energy that is . 
consumed in implementation, how large a part of the user'.s life space is affect- 

22 •■ >. ■ , ; 
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ed, and the amount of change from past .practice that the innovation represents. 
The concept of size makes a great deal of sense 6 to practitioners who must make 
decisions among various innovations and care about Jheir potential influence on 
the total teaching/learning environment. Although the definition of size is not 
oper&tiorial at this time, Texas R&D Center staff were able to reliably, rank 
order 16 different innovations for which Innovation Configuration data had been 
collected. That rank ordering was done in terms of their relative size: As the 
use of innovations is analyzed more closely, perhaps the number of components 
can be used to indicate size, or perhaps a system of weighting components will 
be developed to help in determining how much change is being attempted. 
8. When is an innovation an innovation bundle? 

One of the other concepts that has been useful o is to distinguish between*! 
single innovation and a bundle of innovations. All too frequently evaluators, 
change facilitators, administrators and policy makers refer to a particular 
program as a discrete innovation when in fact it is a bundle of innovations. 
Programs such as Individually Guided Education (IGE) and Competenoy-Based Teach- 
er' Education (CBTE) are innovation bundles, composites of specific innovations 

0 * 

,such as teaming, multi-age grouping, and individualized instruction in vario.us 

■» ■ 

content areas. . . ; 

One of the dilemmas for change facilitators,* researchers and evaluators is 
distinguishing between innovations and innovation bundles. The problem is actu- 
ally more tomplex than that since there is actually a continuum that ranges from 
single component variations--to configuration components--to a particular con- 
figuration to innovation bundles. An interesting question that has to be grap- 
pled with is: When does a component variation becomd a component, become an 
innovation, become an innovation bundle? The concept hf size may have something 
to do with answering this question. 

23 



« » 



. 21 



Implications for Research and Evaluation 

Innovation Configurations as a concept and technique can be used by re- 
search and evaluation staff to describe the present practice of users and non- 
users. It is one way of describing operationally what the treatment is. It can 
be used to document the existence of the treatment in the experimental group and 
absence of components of the treatment in the control group. The concept can be 
used as a dependent variable in studies that .are looking at the causes of use 
and changes in use of a particular innovation. In other situations Innovation 
Configurations can serve as an independent variable when the research questions 
have to do with the relationships between use of the innovation and effects. 

1. Innovation Configurations should be assesed in the treatment and 

Q 

control groups . 

Contrary to past practice it is clear that use of the innovation in the 
treatment g^oup and nonuse of the inriovation in the control group cannot simply 



be assumed. How the innovation is used in eacfry situation and whether the inno- 
vation is used at all must be assejsed^ Innovatign Configurations , provides a 



way to describe that use in operational terms . It is not sufficient to describe 
the implementation requirements, the philosophy or the goals of the innovation. 
For experimental and evaluation studies, it is crucial to describe operationally 
how the treatment has been^used by each user and group of users and to document 
that it ^ is absent operationally in the control settings. Without these data it 
is impossible to have confidence in any findings where use of the innovation or 
treatment isAvariable. 

2. Innovation Configurations can be associated with particular outcomes . 

In future studie^t seems i mp^r t ant to cons i der the relationsh i p between 
par-ficular outcomes and particular components of innovations. It is conceivable 
that certain components of an innovation account for major portions of the out- 
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come variance, while other components may make little or no difference. This 
would probably vary appending upon the outcomes .variables being assessed* There 
also may be interactions between certain components and the outcomes that ar§ 
observed. Thus^multivariate analyses may be appropriate. It also may be, as 

* generic components are identified, that particular components may be associated 
with significant ■effects across different innovations. These may be the most 
powerful determinants of outcomes regardless of which particular innovation 
they are associated with^ Or they may be frequently associated with ease of 
implementation or low implementation of various innovations. Identification of 
these critical components could make a major contribution s to future development 
efforts. 

3. What are the influences that lead to changes in Innovation 
Configurations? 

* During the course of implementation, a multitude of variables may, Influence 
the user to adapt the innovation. Adaptation has emerged as not only a useful 

— -4e^r4 pt^r-^ 

tion to successful implementation. Different conceptual perspectives ancl termi- 
nologies have been proposed to describe an innovation as it undergoes change 

* during implementaiton. One, of course, is that of Innovation Configurations. 
Rice and Rogers (1980) propose the term "re-invention." Miles and Huberman 
(1981) speak of "transformations." 

Probably the most widely known study that has focused on adoption is that 
published in several volumes by the Rand Corporation (Berman & McLaughlin, 
1975). The authors report on their extensive analysis of policy and practice 
relative to the implementation of selected 'fede^ concept for 

the studies is that%of mutual adaptation, which is defined as "an organizational 
process. _in^ijJi_a^ 



realities of the institutional setting, and ,in whichl the .organization changes to 
meet the requirements of the" innovative project (Greenwood, ^ann & McLaughlin, 
1975, p. 31)." ' The Rand studies found such mutual adaptation to' be related to • 
successful implementation. ° 

What causes adaptation? And what effects does adaptation of an innovation 
have on individual users; the. context within which they work, and the clients 
whom tbey serve? These are questions for further research. It appears from 
several studies that change facilitators, such as the principal, can have direct 
influence upon the configurations of the innovation observed in each classroom. 



These effects include consistency and inconsistency of configurations across 
classrooms within a particular buHdin^or a particular school district. In 
addition, user needs, contextual fatffcfrs and client readiness may influence the 
particular configurations of an innovation that are used. n 

Innovation Configurations provides one way to examine the causes of adapta- 
tion as it occurs across sites. Once these adaptations are described and their 



causes ascertained, th t e empirical question becomes whether the outcomes* that are 
associated with the adapted components are changed, enhanced, or reduced as a 
result. Preliminary studies indicate that outcomes are indeed affected by 
change in configuration (Reidy & Hord, .1979) . 

4. What are the relationships between Levels of Use and Innovation 
Configurations* 

The coffcept\of Levels of Use (Hall, Loucks, Rutherford & Newlove, 1975) 
examines whether or not the person is using the innovation. .The Levels of tlse 
decision points emphasize the kinds of adap^ 

making them. 0 However, without configuration information, one doesn/t know what 

* 

is being used. Thus foY* a .complete picture of use, it is useful to assess both 
Levels of Use s and Innovation Configurations in most studies. They are orthogo- 
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nal concepts'. An individual might be at a certain Level of Use, for example 
Routine Use, for any configuration of .an innovation. 

For full documentation of use and nonuse of an innovation in a particular 

study it is important to know first of all whether or not they were users (Is it 

*"■*».'• * 

being used?) and then what configuration of the innovation is being used (What 

*■ * **** 

is it?). Exploratory work indicates that certain configurations of an innova- 
tibn may be associated witH particular Levels of Use, while other configurations 
are found at' aid levels. These *two dimensions together provide an accurate and 
concrete description of the use or honuse of the innovation for each- subject. 
With these data it is^possible to look at the effects of a particular interven- 

* . . 

tion or contextual condition on innovation use, and of a particular configura- 
te 

tion on innovation outcomes. It helps to more clearly understand how treatments 

are actually delivered and wfiat effects are associated with these different 

' ' m • ' • ' 

configurations of delivery. m 

Implications for Change Facilitators 

c 

Change facilitators are individuals who have a responsibility for facili- 

* m 

tating the change process. These include building administrators, district 

administrators, curriculum specialists, staff developers, evaluators, and others 

who ybVk wiflh teachers or other front-line users. °The concept of Innovation 

Configurations has several implications for individuals 0 in this role. 

1. A top i for describin the innovation during dissemination, 
— i — i 2 

In addition to having the description of implementation requirements, 

8 p , 

philosophy, and goals of innovations, it is important in disseminating informa- 
tion about innovations to include descriptive information about what the innova- 
tion looks like in actual practice, that is, an Innovation Configuration de- 
scription. By identifying the essential components of an innovation and accep- 
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table component variations, several objectives can be accomplisljgjfcffn a dissemi- 
nation effort. First, potential adopters are able to compare descriptions of 
the innovation in operation with their present practice. Thus, they can identi- 
fy areas where the components. of the innovation most represent major changes and 

v • 

other areas where the adoption of the innovation will simply reinforce or com- 
plement present practice. This would result in more realistic expectations on 
the part of potential adopters, and in more truthful pfackaging of the innovation 

by innovation agents/ ' 

< 

/ 2. ~ A framework for developing consensus about what "use" means . 

In the implementation of a revised scien.ce program, district staff in 
Jefferson County, Colorado (Welle & Pratt, 1981) developed a twelve component 
checklist to represent the essential components that they felt had to be in „ 

.place for use of that particular innovation* Developing this checklist and then 
sharing it with administrators, teachers, other change facilitators, evaluators 
and researchers resulted in district-wide understanding of what use of the inno- 
vation entailed, avoiding much of the usual ambiguity, uncertainty and confu- 
sion. Unfortunately, this practice does not occur frequently enough. 

Typically, policy makers do not provide configuration information to adop- 
ters even when the use of the innovation is mandated. Thus, there are decrees 
for bilingual education, mainstreaming, busing, etc. without it being clear 
exactly how the innovation is to be operational) zed. In some instances this 
backfires on early adapters, they may 'implement a particular configuration of 
the innovation and three \jjears later, as policy makers^ref ine their thinking 
about acceptable use, the early adopters find themselves "out of compliance" 
with their early configurations of the innovation. In addition to punishing 
early adopters this occurrence reinforges and rewards late adopters for delaying 
their decision' to implement new practices. ' 
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Again, early development of a consensus about what use of the innovation" 

m 

entails can lead to consistency in communication, clarification of terminology 
and a much easier implementation effort since all would know where they are 
headed and have some clarity in how to approach getting there. 

3. A framework for focusing the delivery of assistance to innovation 

I 

users. 

Assessment of Innovation Configurations could lead to particular workshops 
that are targeted toward particular Innovation Configuration components (Hall & 
Loucks, 1981). For example, if it is desirable that a particular grouping vari- 
ation be used, a workshop could be developed and delivered that would provide 
users skill in this particular component variation. On the other band, if all 
teachers were using tfie same component variation and more variety was desired, 
then set -breaking workshops could be conducted to help expand the number of 
configuration* variations that were being used. If change facilitators were to 
focus on particular components, they could more effectively organize and target 
their interventions. For example, principals could focus classroom observations 
on particular' components rather than attempting to look at the whole complex 
array of an innovation (Melle & Pratt, 1981). 

4. Phasing the implementation of components may be desirable . 

All too frequently, innovation implementation occurs with a "big plunge." 
As of some particular date ajl users are expected to use the entire innovation 
or innovation bundle. »What often happens in this case, particularly with large 
innovations, is that, the plunge literally puts everyone in over their heads. 
As a* result they gradually build a distaste for their experiences with the inno- 
vation and withdraw from use of its major components. 

With more complex and larger innovations, or ones which require' a great 
deal of change from previous practice, a phased implementation might be more 
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effective. One or two innovation components might be identified for implementa- 
tion during the first year, and then in subsequent years additional major compo- 
nents of the innovation could be implemented. This would allow limited staff 
development resources and user energy to be focused upon making the first compo- 
n ents o p eration al before they are overwhelmed with the addition of other equally 
complex components. This approach would acknowledge that change is a process. 
By phasing use of components more successful implementation efforts can be 
developed. This also means that the evaluators will need to acknowledge limited 
use during the first year., and postpone summative evaluation until implementa- 
tion of all components has occurred. This idea of phasing the implementation of 
components also suggests that "competence" would have a different definition for 
.first year users than it would for third year users. 

In Summary 

The concept of Innovation Configurations has proven very useful to staff at 
the Texas R&D Center and practitioner colleagues in grappling with defining dif- 
ferent innovations and determining whether or not they are actually being used 
in classrooms. Clearly much definitional work is necessary to determine Innova- 
tion Configuration components and to develop component checklists. However, it 
results in increased clarity and specificity about what the innovation is. This 
can be of great utility in change process research and, as illustrated in the 
other papers in the symposium, can be of use to evaluators and change facilita- 
tors as they attempt to be more effective in their work. 

Present research at the Texas R&D Center is looking particularly at the 
effects of Change facilitators upon the configurations of users. Others are 
invited to consider the concept, to help clarify the thinking that has occurred 
and to make further contributions. • 
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